QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game

A rise in Advanced Persistent Threats (APTs) has introduced a need for robustness against long-running, stealthy attacks which circumvent existing cryptographic security guarantees. FlipIt is a security game that models attacker-defender interactions in advanced scenarios such as APTs. Previous work analyzed extensively non-adaptive strategies in FlipIt, but adaptive strategies rise naturally in practical interactions as players receive feedback during the game. We model the FlipIt game as a Markov Decision Process and introduce QFlip, an adaptive strategy for FlipIt based on temporal difference reinforcement learning. We prove theoretical results on the convergence of our new strategy against an opponent playing with a Periodic strategy. We confirm our analysis experimentally by extensive evaluation of QFlip against specific opponents. QFlip converges to the optimal adaptive strategy for Periodic and Exponential opponents using associated state spaces. Finally, we introduce a generalized QFlip strategy with composite state space that outperforms a Greedy strategy for several distributions including Periodic and Uniform, without prior knowledge of the opponent's strategy. We also release an OpenAI Gym environment for FlipIt to facilitate future research.Comment: Outstanding Student Paper awar

Strategic Network Formation with Attack and Immunization

Strategic network formation arises where agents receive benefit from connections to other agents, but also incur costs for forming links. We consider a new network formation game that incorporates an adversarial attack, as well as immunization against attack. An agent's benefit is the expected size of her connected component post-attack, and agents may also choose to immunize themselves from attack at some additional cost. Our framework is a stylized model of settings where reachability rather than centrality is the primary concern and vertices vulnerable to attacks may reduce risk via costly measures. In the reachability benefit model without attack or immunization, the set of equilibria is the empty graph and any tree. The introduction of attack and immunization changes the game dramatically; new equilibrium topologies emerge, some more sparse and some more dense than trees. We show that, under a mild assumption on the adversary, every equilibrium network with $n$ agents contains at most $2n-4$ edges for $n\geq 4$. So despite permitting topologies denser than trees, the amount of overbuilding is limited. We also show that attack and immunization don't significantly erode social welfare: every non-trivial equilibrium with respect to several adversaries has welfare at least as that of any equilibrium in the attack-free model. We complement our theory with simulations demonstrating fast convergence of a new bounded rationality dynamic which generalizes linkstable best response but is considerably more powerful in our game. The simulations further elucidate the wide variety of asymmetric equilibria and demonstrate topological consequences of the dynamics e.g. heavy-tailed degree distributions. Finally, we report on a behavioral experiment on our game with over 100 participants, where despite the complexity of the game, the resulting network was surprisingly close to equilibrium.Comment: The short version of this paper appears in the proceedings of WINE-1

Socially Optimal Mining Pools

Mining for Bitcoins is a high-risk high-reward activity. Miners, seeking to reduce their variance and earn steadier rewards, collaborate in pooling strategies where they jointly mine for Bitcoins. Whenever some pool participant is successful, the earned rewards are appropriately split among all pool participants. Currently a dozen of different pooling strategies (i.e., methods for distributing the rewards) are in use for Bitcoin mining. We here propose a formal model of utility and social welfare for Bitcoin mining (and analogous mining systems) based on the theory of discounted expected utility, and next study pooling strategies that maximize the social welfare of miners. Our main result shows that one of the pooling strategies actually employed in practice--the so-called geometric pay pool--achieves the optimal steady-state utility for miners when its parameters are set appropriately. Our results apply not only to Bitcoin mining pools, but any other form of pooled mining or crowdsourcing computations where the participants engage in repeated random trials towards a common goal, and where "partial" solutions can be efficiently verified

Threshold FlipThem:when the winner does not need to take all

We examine a FlipIt game in which there are multiple resources which a monolithic attacker is trying to compromise. This extension to FlipIt was considered in a paper in GameSec 2014, and was there called FlipThem. Our analysis of such a situation is focused on the situation where the attacker’s goal is to compromise a threshold of the resources. We use our game theoretic model to enable a defender to choose the correct configuration of resources (number of resources and the threshold) so as to ensure that it makes no sense for a rational adversary to try to attack the system. This selection is made on the basis of the relative costs of the attacker and the defender

Investing in Prevention or Paying for Recovery - Attitudes to Cyber Risk

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Broadly speaking an individual can invest time and effort to avoid becoming victim to a cyber attack and/or they can invest resource in recovering from any attack. We introduce a new game called the pre-vention and recovery game to study this trade-off. We report results from the experimental lab that allow us to categorize different approaches to risk taking. We show that many individuals appear relatively risk loving in that they invest in recovery rather than prevention. We find little difference in behavior between a gain and loss framing

Multi-rate Threshold FlipThem

A standard method to protect data and secrets is to apply threshold cryptography in the form of secret sharing. This is motivated by the acceptance that adversaries will compromise systems at some point; and hence using threshold cryptography provides a defence in depth. The existence of such powerful adversaries has also motivated the introduction of game theoretic techniques into the analysis of systems, e.g. via the FlipIt game of van Dijk et al. This work further analyses the case of FlipIt when used with multiple resources, dubbed FlipThem in prior papers. We examine two key extensions of the FlipThem game to more realistic scenarios; namely separate costs and strategies on each resource, and a learning approach obtained using so-called fictitious play in which players do not know about opponent costs, or assume rationality

Estimating Systematic Risk in Real-World Networks

Abstract. Social, technical and business connections can all give rise to security risks. These risks can be substantial when individual compro-mises occur in combinations, and difficult to predict when some connec-tions are not easily observed. A significant and relevant challenge is to predict these risks using only locally-derivable information. We illustrate by example that this challenge can be met if some general topological features of the connection network are known. By simulat-ing an attack propagation on two large real-world networks, we identify structural regularities in the resulting loss distributions, from which we can relate various measures of a network’s risks to its topology. While de-riving these formulae requires knowing or approximating the connective structure of the network, applying them requires only locally-derivable information. On the theoretical side, we show that our risk-estimating methodology gives good approximations on randomly-generated scale-free networks with parameters approximating those in our study. Since many real-world networks are formed through preferential attachment mechanisms that yield similar scale-free topologies, we expect this methodology to have a wider range of applications to risk management whenever a large number of connections is involved

Post-incident audits on cyber insurance discounts

We introduce a game-theoretic model to investigate the strategic interaction between a cyber insurance policyholder whose premium depends on her self-reported security level and an insurer with the power to audit the security level upon receiving an indemnity claim. Audits can reveal fraudulent (or simply careless) policyholders not following reported security procedures, in which case the insurer can refuse to indemnify the policyholder. However, the insurer has to bear an audit cost even when the policyholders have followed the prescribed security procedures. As audits can be expensive, a key problem insurers face is to devise an auditing strategy to deter policyholders from misrepresenting their security levels to gain a premium discount. This decision-making problem was motivated by conducting interviews with underwriters and reviewing regulatory filings in the US; we discovered that premiums are determined by security posture, yet this is often self-reported and insurers are concerned by whether security procedures are practised as reported by the policyholders. To address this problem, we model this interaction as a Bayesian game of incomplete information and devise optimal auditing strategies for the insurers considering the possibility that the policyholder may misrepresent her security level. To the best of our knowledge, this work is the first theoretical consideration of post-incident claims management in cyber security. Our model captures the trade-off between the incentive to exaggerate security posture during the application process and the possibility of punishment for non-compliance with reported security policies. Simulations demonstrate that common sense techniques are not as efficient at providing effective cyber insurance audit decisions as the ones computed using game theory